Determining stereo distance information using imaging devices integrated into propeller blades

ABSTRACT

A propeller provided on an aerial vehicle may include a digital camera or other imaging device embedded into a surface of one of the blades of the propeller. The digital camera may capture images while the propeller is rotating at an operational speed. Images captured by the digital camera may be processed to recognize one or more objects therein, and to determine ranges to such objects by stereo triangulation techniques. Using such ranges, a depth map or other model of the surface features in an environment in which the aerial vehicle is operating may be defined and stored or used for any purpose. A propeller may include digital cameras or other imaging devices embedded into two or more blades, and may also use such images to determine ranges to objects by stereo triangulation techniques.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/243,844, now U.S. Pat. No. 10,033,980, filed Aug. 22, 2016, thecontents of which are incorporated by reference herein in theirentirety.

BACKGROUND

Many aerial vehicles such as airplanes, helicopters or other airshipsare configured to operate in two or more flight modes, including aforward flight mode (or a substantially horizontal flight mode) in whichthe aerial vehicle travels from one point in space (e.g., a land-basedpoint or, alternatively, a sea-based or air-based point) to anotherpoint by traveling over at least a portion of the Earth. An aerialvehicle may also be configured to engage in a vertical flight mode inwhich the aerial vehicle travels in a vertical or substantially verticaldirection from one altitude to another altitude (e.g., upward ordownward, from a first point on land, on sea or in the air to a secondpoint in the air, or vice versa) substantially normal to the surface ofthe Earth, or hovers (e.g., maintains a substantially constantaltitude), with an insubstantial change in horizontal or lateralposition. An aerial vehicle may be further configured to engage in bothforward and vertical flight modes, e.g., in a hybrid mode in which aposition of the aerial vehicle changes in both horizontal and verticaldirections. Forces of lift and thrust are commonly applied to aerialvehicles using one or more propellers, or devices having blades that aremounted about a hub and joined to a shaft or other component of a primemover, which may rotate at angular velocities of thousands ofrevolutions per minute during flight operations.

Aerial vehicles (including, specifically, unmanned aerial vehicles, orUAVs) are frequently equipped with one or more imaging devices such asdigital cameras which may be used to aid in the guided or autonomousoperation of an aerial vehicle, to determine when the aerial vehicle hasarrived at or passed over a given location, or is within range of one ormore structures, features, objects or humans (or other animals), toconduct surveillance or monitoring operations, or for any other purpose.Outfitting an aerial vehicle with one or more imaging devices typicallyrequires installing housings, turrets or other structures or features bywhich the imaging devices may be mounted to the aerial vehicle. Suchstructures or features add weight to the aerial vehicle, and mayincrease the amount or extent of drag encountered during flight, therebyexacting an operational cost from the aerial vehicle in exchange for themany benefits that imaging devices may provide.

Stereo ranging (or stereo triangulation) is a process by which distancesor ranges to objects may be determined from digital images depictingsuch objects that are captured using imaging devices, such as digitalcameras, that are separated by a fixed distance. For example, byprocessing pairs of images of an environment that are captured byimaging devices, ranges to points expressed in both of the images(including but not limited to points associated with specific objects)may be determined by finding a virtual intersection of pairs of linesextending from the respective lenses or sensors of the imaging devicesthrough representations of such points within each of the images. Ifeach of the images of the environment is captured substantiallysimultaneously, or if conditions of the environment are substantiallyunchanged when each of the images is captured, a range to a single pointwithin the environment at a given time may be determined based on abaseline distance between the lenses or sensors of the imaging devicesthat captured such images and a disparity, or a distance betweencorresponding representations of a single point in space expressedwithin both of the images when the images are superimposed upon oneanother. Such processes may be completed for any number of points inthree-dimensional space that are expressed in both of the images, and amodel of such points, e.g., a point cloud, a depth map or a depth model,may be defined accordingly. The model of such points may be updated aspairs of images are subsequently captured and processed to determineranges to such points.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A through 1E are views of aspects of one system for determiningstereo distance information using an imaging device integrated into apropeller blade in accordance with embodiments of the presentdisclosure.

FIG. 2 is a block diagram of one system for determining stereo distanceinformation using imaging devices integrated into propeller blades inaccordance with embodiments of the present disclosure.

FIG. 3 is a flow chart of one process for determining stereo distanceinformation using imaging devices integrated into propeller blades inaccordance with embodiments of the present disclosure.

FIGS. 4A, 4B and 4C are views of aspects of one system for determiningstereo distance information using imaging devices integrated into apropeller blade in accordance with embodiments of the presentdisclosure.

FIGS. 5A, 5B and 5C are views of aspects of one system for determiningstereo distance information using imaging devices integrated intopropeller blades in accordance with embodiments of the presentdisclosure.

FIG. 6 is a flow chart of one process for determining stereo distanceinformation using imaging devices integrated into propeller blades inaccordance with embodiments of the present disclosure.

FIG. 7 is a view of aspects of one system for determining stereodistance information using imaging devices integrated into a propellerblade in accordance with embodiments of the present disclosure.

FIG. 8A and FIG. 8B are views of propeller blades having imaging devicesintegrated therein for determining stereo distance information inaccordance with embodiments of the present disclosure.

FIGS. 9A through 9D are views of aspects of one system for determiningstereo distance information using imaging devices integrated intopropeller blades in accordance with embodiments of the presentdisclosure.

FIGS. 10A, 10B and 10C are views of aspects of one system fordetermining stereo distance information using imaging devices integratedinto propeller blades in accordance with embodiments of the presentdisclosure.

DETAILED DESCRIPTION

As is set forth in greater detail below, the present disclosure isdirected to determining ranges or distances from operating aerialvehicles to one or more objects. More specifically, the systems andmethods disclosed herein are directed to determining stereo distanceinformation using imaging devices (e.g., digital cameras) that have beenintegrated into blades of one or more operational propellers. Theimaging devices may be digital cameras (e.g., black-and-white, grayscaleor color cameras) or any other devices for capturing and interpretinglight that is reflected from one or more objects. In some embodiments,an imaging device may be embedded, installed or otherwise integratedinto a surface of a blade of a propeller for providing lift or thrust toan aerial vehicle, with the imaging device provided at a predetermineddistance (e.g., a fixed radius) from a hub of the propeller. When thepropeller blade is rotating at an operating angular velocity, e.g.,thousands of revolutions per minute or more, a first image may becaptured using the imaging device with the propeller blade at a firstangular orientation, and a second image may be captured with thepropeller blade at a second angular orientation. The first image and thesecond image may be aligned with respect to one another, and ranges toobjects expressed in each of the images may be determined according toone or more stereo ranging algorithms or techniques.

Where an operating angular velocity of a propeller blade is sufficientlyhigh, a single imaging device integrated into the propeller blade mayact as two imaging devices, by swinging from a first position in spaceto a second position in space at within a fraction of a second, and thefirst image captured by the imaging device in the first position and asecond image captured by the imaging device in the second position maybe determined to have been captured substantially simultaneously. Forexample, where a first angular orientation of a blade of a propeller anda second angular orientation of the blade are separated by approximatelyone hundred eighty degrees (180°), e.g., an opposite direction angle, abaseline distance or separation equal to twice a predetermined distanceor radius between a hub of the propeller and an imaging device embeddedinto the blade may be used to calculate ranges to objects expressed inthe each of the images. In other embodiments, imaging devices may beintegrated into the same blade of a propeller, or into two or more ofthe blades of the propeller, at equal or different radii from a hub.Images captured by such devices during the rotation of such propellersmay be aligned with respect to one another, and ranges to objectsexpressed in each of such images may be determined using stereotriangulation, e.g., using one or more computer-based stereo rangingalgorithms or techniques.

Referring to FIGS. 1A through 1E, a system for determining stereodistance information using an imaging device integrated into a propellerblade in accordance with embodiments of the present disclosure is shown.As is shown in FIG. 1A, an aerial vehicle 110 includes a control center120, a plurality of motors 130-1, 130-2, 130-3, 130-4 and a plurality ofpropellers 140-1, 140-2, 140-3, 140-4, with each of the propellers140-1, 140-2, 140-3, 140-4 rotatably coupled to one of the motors 130-1,130-2, 130-3, 130-4. The propeller 140-4 includes a hub 142-4, a firstblade 144-4 and a second blade 146-4, with an imaging device 150integrated into an underside of the first blade 144-4 at a radius r fromthe hub 142-4. As is shown in FIG. 1A, the first blade 144-4 is alignedat an angle θ₁ at time t₁.

As is shown in FIG. 1B, the aerial vehicle 110 captures an image 10-1while the propeller 140-4 is rotating, e.g., under power of the motor130-4, and with the imaging device 150 oriented substantially downwardlyat time t₁, as the first blade 144-4 is aligned at the angle θ₁. Forexample, the imaging device 150 may be configured to capture color orgrayscale images of ground-based features in the area in which theimaging device 150 operates (e.g., structures, vehicles or othermachines, plant or animal life), or airborne elements that may approachor be located near the aerial vehicle 110 (e.g., birds, other aerialvehicles, or any other airborne objects).

As is shown in FIG. 1C, the first blade 144-4 has completed one half ofone revolution and is aligned at an angle θ₂ at time t₂. For example,where the propeller 140-4 is spinning at an operational angular velocityof approximately three thousand revolutions per minute (3000 rpm), thefirst blade 144-4 will revolve from the angle θ₁ to the angle θ₂ in anelapsed time (e.g., t₂−t₁) of one six-thousands of a minute ( 1/6000min), or one one-hundredth of one second (0.01 sec). As is shown in FIG.1D, while the propeller 140-4 is rotating, and with the first blade144-4 aligned at the angle θ₂, the aerial vehicle 110 captures an image10-2 with the imaging device 150 oriented substantially downwardly attime t₂. A rotated image 10-2′ that coincides with the image 10-1 may beobtained by rotating the image 10-2 by a difference Δθ between the angleθ₂ and the angle θ₁, or θ₂−θ₁.

As is discussed above, pairs of images that are captured by one or moreimaging devices integrated into surfaces of propellers may be co-alignedand subjected to one or more stereo ranging analyses, in order todetermine ranges to any number of points that are expressed in both ofthe images. For example, ranges to a plurality of points within anenvironment that appear in each of the images may be combined to form apoint cloud, a depth map or another representation of athree-dimensional profile of the environment. As is shown in FIG. 1E,the image 10-1 and the rotated image 10-2′ may be provided to a computerdevice 112 for processing. The computer device 112 may reside on theaerial vehicle 110 or in one or more external locations, including aground-based or a “cloud”-based facility having one or more servers orother computer devices, a facility residing aboard one or more otheraerial vehicles (not shown), or a facility in any other location. Thecomputer device 112 may fuse together the features of the images 10-1,10-2′ captured by the imaging device 150 at times t₁ and t₂, which areseparated by a fraction of a second, and determine which pointsexpressed in the image 10-1 correspond to points expressed in therotated image 10-2′. Distances to points corresponding to such featuresmay be determined according to stereo ranging algorithms or techniquesand stored in one or more data stores or used for any purpose, includingbut not limited to navigation, guidance, surveillance or collisionavoidance.

For example, as is shown in FIG. 1E, a depth map 15 of average ornominal ranges to regions corresponding to features below the aerialvehicle 110 that are expressed in both the image 10-1 and the rotatedimage 10-2′, and tolerances associated with such ranges, may begenerated and stored in one or more data stores. The depth map 15 ofFIG. 1E includes ranges to a region 15-1 corresponding to a firstautomobile (e.g., approximately one hundred nineteen feet), a region15-2 corresponding to a street on which the first automobile travels(e.g., approximately one hundred twenty-two and one half feet), a region15-3 corresponding to a second automobile on the street (e.g.,approximately one hundred eighteen feet) a region 15-4 corresponding toa dwelling (e.g., approximately one hundred one feet), a region 15-5corresponding to a tree (e.g., approximately eighty-six feet), a region15-6 corresponding to a pet (e.g., approximately one hundred twenty-twofeet) and a region 15-7 generally corresponding to a ground area (e.g.,approximately one hundred twenty-four feet) not occupied or covered bythe dwelling, the tree or the pet. The depth map 15 may be used for anypurpose, including but not limited to identifying a suitably large, flatand sturdy landing site that may accommodate one or more dimensions ofthe aerial vehicle 110.

Accordingly, the systems and methods of the present disclosure aredirected to determining stereo distance information using imagingdevices that are integrated into propeller blades on operating aerialvehicles. The propellers may include any number of blades (e.g., twoblades, such as the propeller 140-4 of FIGS. 1A through 1D, as well asthree, four or more blades) mounted about a hub that is configured toreceive a mast or a shaft of a transmission associated with a motor, andto be rotated about the mast or shaft by the motor at a desired angularvelocity for providing forces of lift or thrust to the aerial vehicle.Any number of the blades may include any number of imaging devices thatare integrated therein, e.g., the single imaging device 150 embeddedinto the blade 144-4 of FIGS. 1A through 1D, or any number of otherimaging devices.

In accordance with the present disclosure, imaging devices that areintegrated into blades of operating propellers, and images captured bysuch imaging devices, may be used to determine stereo distanceinformation according to any number of stereo ranging algorithms ortechniques. Outputs from such algorithms or techniques may be generatedor stored in any form, and used for any purpose. For example, in someembodiments, distances to objects or features in an environmentdetermined according to stereo ranging algorithms or techniques may beaggregated into a depth map, such as the depth map 15 of FIG. 1E, thatidentifies or represents nominal or average distances to such objects orfeatures and tolerances associated with such distances.

In some other embodiments, a point cloud or other three-dimensionalrepresentation of an environment may be generated and stored in one ormore data files. The point cloud may represent positions of each of thepoints that appear in both of the images of a pair, with pixel-levelresolution. The high-speed, reliably repetitive nature of a rotatingpropeller blade enables data to be captured regarding ranges to suchpoints at high rates of speed, thereby enabling tolerances or confidencelevels associated with such positions to be narrowed considerably afteronly a number of images are captured, over a brief period of time.

Imaging devices may be integrated into blades of a propeller in anymanner, e.g., by embedding an imaging device into a blade, or byadhering an imaging device to a surface of a blade, in accordance withthe present disclosure. Imaging devices that are integrated into suchblades may have a field of view or axis of orientation that are alignednormal to the surfaces of such blades, or at any other angle ororientation. In some embodiments, the imaging devices may haveadjustable fields of view or axes of orientation, e.g., by one or moreactuated or motorized features for adjusting either a focal length or anangular orientation of the imaging device. Additionally, the imagingdevices may be integrated into a blade of a propeller at any radius froma hub of the propeller. Similarly, an aerial vehicle having one or moreimaging devices integrated into propellers may further include one ormore additional imaging devices that integrated into portions of theaerial vehicle that are fixed in orientation, e.g., to a fuselage orother non-rotating portion of the aerial vehicle, and such imagingdevices may be used in concert with integrated imaging devices inranging applications.

Imaging data (e.g., visual imaging data) may be captured using one ormore imaging devices such as digital cameras. Such devices may generallyoperate by capturing light that is reflected from objects, and bysubsequently calculating or assigning one or more quantitative values toaspects of the reflected light, e.g., pixels, generating an output basedon such values, and storing such values in one or more data stores.Digital cameras may include one or more sensors having one or morefilters associated therewith, and such sensors may detect informationregarding aspects of any number of pixels of the reflected lightcorresponding to one or more base colors (e.g., red, green or blue) ofthe reflected light. Such sensors may generate data files including suchinformation, e.g., digital images, and store such data files in one ormore onboard or accessible data stores (e.g., a hard drive or other likecomponent), as well as one or more removable data stores (e.g., flashmemory devices), or displayed on one or more broadcast or closed-circuittelevision networks, or over a computer network as the Internet.

A digital image is a collection of pixels, typically arranged in anarray, which defines an optically formed reproduction of one or moreobjects, backgrounds or other features of a scene and may be stored in adata file. In a visual image, each of the pixels represents oridentifies a color or other light condition associated with a portion ofsuch objects, backgrounds or features. For example, a black-and-whitevisual image includes a single bit for representing a light condition ofthe pixel in a binary fashion (e.g., either black or white), while agrayscale visual image may represent the light condition in multiplebits (e.g., two to eight bits for defining tones of gray in terms ofpercentages or shares of black-and-white), and a color visual image mayinclude groups of bits corresponding to each of a plurality of basecolors (e.g., red, green or blue), and the groups of bits maycollectively represent a color associated with the pixel. A depth imageis also a collection of pixels that defines an optically formedreproduction of one or more objects, backgrounds or other features of ascene, and may also be stored in a data file. Unlike the pixels of avisual image, however, each of the pixels of a depth image represents oridentifies not a light condition or color of such objects, backgroundsor features, but a distance to objects, backgrounds or features. Forexample, a pixel of a depth image may represent a distance between asensor of an imaging device that captured the depth image (e.g., a depthcamera or range sensor) and the respective object, background or featureto which the pixel corresponds.

Imaging data files that are stored in one or more data stores may beprinted onto paper, presented on one or more computer displays, orsubjected to one or more analyses, such as to identify items expressedtherein. Such data files may be stored in any number of formats,including but not limited to .JPEG or .JPG files, or GraphicsInterchange Format (or “.GIF”), Bitmap (or “.BMP”), Portable NetworkGraphics (or “.PNG”), Tagged Image File Format (or “.TIFF”) files, AudioVideo Interleave (or “.AVI”), QuickTime (or “.MOV”), Moving PictureExperts Group (or “.MPG,” “.MPEG” or “.MP4”) or Windows Media Video (or“.WMV”) files.

Reflected light may be captured or detected by an imaging device if thereflected light is within the device's field of view, which is definedas a function of a distance between a sensor and a lens within thedevice, viz., a focal length, as well as a location of the device and anangular orientation of the device's lens. Accordingly, where an objectappears within a depth of field, or a distance within the field of viewwhere the clarity and focus is sufficiently sharp, an imaging device maycapture light that is reflected off objects of any kind to asufficiently high degree of resolution using one or more sensorsthereof, and store information regarding the reflected light in one ormore data files.

Many imaging devices also include manual or automatic features formodifying their respective fields of view or orientations. For example,a digital camera may be configured in a fixed position, or with a fixedfocal length (e.g., fixed-focus lenses) or angular orientation.Alternatively, an imaging device may include one or more actuated ormotorized features for adjusting a position of the imaging device, orfor adjusting either the focal length (e.g., zooming the imaging device)or the angular orientation (e.g., the roll angle, the pitch angle or theyaw angle), by causing a change in a distance between the sensor and thelens (e.g., optical zoom lenses or digital zoom lenses), a change in alocation of the imaging device, or a change in one or more of the anglesdefining an angular orientation.

For example, an imaging device may be hard-mounted to a support ormounting that maintains the device in a fixed configuration or anglewith respect to one, two or three axes. Alternatively, however, animaging device may be provided with one or more motors and/orcontrollers for manually or automatically operating one or more of thecomponents, or for reorienting the axis or direction of the device,i.e., by panning or tilting the device. Panning an imaging device maycause a rotation within a horizontal plane or about a vertical axis(e.g., a yaw), while tilting an imaging device may cause a rotationwithin a vertical plane or about a horizontal axis (e.g., a pitch).Additionally, an imaging device may be rolled, or rotated about its axisof rotation, and within a plane that is perpendicular to the axis ofrotation and substantially parallel to a field of view of the device.

Some modern imaging devices may digitally or electronically adjust animage identified in a field of view, subject to one or more physical andoperational constraints. For example, a digital camera may virtuallystretch or condense the pixels of an image in order to focus or broadenthe field of view of the digital camera, and also translate one or moreportions of images within the field of view. Imaging devices havingoptically adjustable focal lengths or axes of orientation are commonlyreferred to as pan-tilt-zoom (or “PTZ”) imaging devices, while imagingdevices having digitally or electronically adjustable zooming ortranslating features are commonly referred to as electronic PTZ (or“ePTZ”) imaging devices.

Information and/or data regarding features or objects expressed inimaging data, including colors, textures or outlines of the features orobjects, may be extracted from the data in any number of ways. Forexample, colors of pixels, or of groups of pixels, in a digital imagemay be determined and quantified according to one or more standards,e.g., the RGB (“red-green-blue”) color model, in which the portions ofred, green or blue in a pixel are expressed in three correspondingnumbers ranging from 0 to 255 in value, or a hexadecimal model, in whicha color of a pixel is expressed in a six-character code, wherein each ofthe characters may have a range of sixteen. Colors may also be expressedaccording to a six-character hexadecimal model, or # NNNNNN, where eachof the characters N has a range of sixteen digits (i.e., the numbers 0through 9 and letters A through F). The first two characters NN of thehexadecimal model refer to the portion of red contained in the color,while the second two characters NN refer to the portion of greencontained in the color, and the third two characters NN refer to theportion of blue contained in the color. For example, the colors whiteand black are expressed according to the hexadecimal model as # FFFFFFand #000000, respectively, while the color candy apple red is expressedas #31314A. Any means or model for quantifying a color or color schemawithin an image or photograph may be utilized in accordance with thepresent disclosure. Moreover, textures or features of objects expressedin a digital image may be identified using one or more computer-basedmethods, such as by identifying changes in intensities within regions orsectors of the image, or by defining areas of an image corresponding tospecific surfaces.

Furthermore, edges, contours, outlines, colors, textures, silhouettes,shapes or other characteristics of objects, or portions of objects,expressed in still or moving digital images may be identified using oneor more algorithms or machine-learning tools. The objects or portions ofobjects may be stationary or in motion, and may be identified at single,finite periods of time, or over one or more periods or durations. Suchalgorithms or tools may be directed to recognizing and markingtransitions (e.g., the edges, contours, outlines, colors, textures,silhouettes, shapes or other characteristics of objects or portionsthereof) within the digital images as closely as possible, and in amanner that minimizes noise and disruptions, and does not create falsetransitions. Some detection algorithms or techniques that may beutilized in order to recognize characteristics of objects or portionsthereof in digital images in accordance with the present disclosureinclude, but are not limited to, Canny edge detectors or algorithms;Sobel operators, algorithms or filters; Kayyali operators; Roberts edgedetection algorithms; Prewitt operators; Frei-Chen methods; or any otheralgorithms or techniques that may be known to those of ordinary skill inthe pertinent arts.

As is discussed above, the systems and methods of the present disclosureare directed to determining stereo distance information using imagingdevices that are integrated into propeller blades on operating aerialvehicles. The images captured by such imaging devices may be processedaccording to one or more stereo ranging algorithms or techniques.Although determining depth information from a dynamic environment usingby such algorithms or techniques typically requires the use of at leasttwo imaging devices that are separated by a baseline distance, and thecapture of imaging data from such imaging devices substantiallysimultaneously, the systems and methods of the present disclosure may,in some embodiments, use imaging data captured a single imaging deviceembedded in a propeller blade for stereo ranging. The imaging device maycapture imaging data with the propeller blade at different orientations,thereby relying on the typically high rotational speed of an aerialvehicle's propeller to effectively position the imaging device in twoplaces at once.

For example, an imaging device that may capture images at frame rates onthe order of hundreds of frames per second (fps), and is embedded into asurface of a propeller blade that is rotating at angular velocities onthe order of thousands of revolutions per minute (rpm), may captureclear images with the propeller at different orientations and processsuch images to make depth determinations regarding any objects that areexpressed in both of the images. In particular, where an imaging devicemay be configured to capture images with the propeller at orientationsthat are approximately one hundred eighty degrees, or 180°, apart oropposed from one another, a baseline distance or separation (e.g., twicethe radius of the imaging device from a hub of the propeller), adisparity (e.g., a distance between a common point in each of theimages), a focal length of the imaging device and the contents of therespective images may be processed in order to determine ranges to eachof the objects expressed in the two images, and to define a depth map, adepth model, or another depth image of an environment accordingly.

Distances (or depths or ranges) to objects that are represented in apair of stereo images captured by imaging devices (e.g., digitalcameras) having fields of view that overlap, at least partially. Foreach point of each object that appears in both of the images, linesextending from the respective lenses, lens modules or other sensors ofthe respective imaging devices through representations of the points ofthe objects in each of the images will virtually intersect at a locationcorresponding to the actual position of that point, in three-dimensionalspace. Through the use of traditional geometric principles andproperties, e.g., the properties of similar triangles, as well as theknown or knowable variables such as baseline distance or separationbetween the imaging devices, the disparity between the points within therespective images and the focal lengths of the respective imagingdevices, coordinates of the intersecting point may be determinedaccordingly.

Because a propeller of an aerial vehicle typically rotates at angularvelocities of several thousand revolutions per minute, embedding asingle imaging device into a propeller blade, e.g., into a surface of apropeller, enables stereo images to be captured with by a single imagingdevice at known positions and at given times. In order to determinestereo distance information from a pair of images, each surface pointthat is visible within a first one of the images must be identified inthe second one of the images, and the geometric position of the imagingdevice as each of the images was captured must be known. Representationsof a common point within two stereo images are sometimes calledepipoles, or a conjugate pair of such epipoles, and the disparity isdefined as the distance between the conjugate pair of epipoles when thetwo images are superimposed.

Where a point in space appears in two images, e.g., as epipoles, a planedefined by the positions of the respective epipoles within the imagesand an actual position of the point in space is called an epipolarplane. The images may then be co-aligned based on their contents, e.g.,along lines corresponding to intersections of the epipolar plane withthe respective image planes, or their respective epipolar lines. Afterthe images have been aligned based on their contents, an actual positionof the object may be determined by triangulating lines extending fromlenses, lens modules or other sensors of an imaging device through therepresentations of the points in the respective images within theimaging plane. An intersection of such lines corresponds to the actualposition of the point, and a distance to the point may be determinedaccordingly based on this actual position. Stereo ranging algorithms andtechniques may be used to determine ranges or distances to each of thepoints that appears in both of the images, and such ranges or distancesmay be used to define a point cloud, a depth map or anotherthree-dimensional model of the environment in which the object isprovided. The depth model may be stored in a data file (e.g., a depthimage) or utilized for any purpose, including but not limited tonavigation, guidance, surveillance or collision avoidance.

Stereo ranging algorithms and techniques thus require determiningcorrespondences of the epipoles in each of the pair of images, with eachof the epipoles corresponding to a common point in three-dimensionalspace. When a plurality of correspondences of epipoles are identifiedfrom each of a pair of images of a scene, disparities for each of theconjugate pairs of epipoles may be determined, and a map of suchdisparities that mimics a three-dimensional structure of the scene maybe reconstructed accordingly if information regarding aspects of thescene, e.g., geometric parameters such as the baseline distance orseparation, the focal lengths of the imaging devices and others, isknown.

There are a number of computer-based stereo ranging algorithms andtechniques for determining real-world positions of points expressed inpairs of images of scenes, and for generating depth maps, point cloudsor other three-dimensional representations of such scenes based on suchpositions. Such algorithms or techniques may aid in the performance ofcalibration, correspondence and/or reconstruction functions. Forexample, the Open Source Computer Vision (or “OpenCV”) library includesa number of computer-based algorithms or other programming functionsthat are directed to determining distances or ranges from pairs ofimages. Similarly, a number of other stereo ranging algorithms ortechniques programmed in the MATLAB language are publicly available.Computer-based algorithms or techniques are available from a number ofother sources, as well.

Imaging devices may be integrated into propellers that are alignedhorizontally or vertically, e.g., in forward or aft orientations, or inupward or downward orientations, or at any other orientations or angles,which may be relative or absolute. In some embodiments, two or moredigital cameras may be integrated into a propeller, either in the sameblade, or in different blades. The digital cameras may be homogenous(e.g., functionally equivalent or having the same capacities) or,alternatively, heterogeneous (e.g., having different capacities), andstereo images captured by such cameras for determining depths may beprocessed in multiple calculations. In some embodiments, an aerialvehicle may include one or more imaging devices that are integrated intoblades of a rotating propeller and also mounted to non-rotating featuresof the aerial vehicle. Images captured by each of the imaging devicesmay be used for stereo ranging purposes, e.g., by determining baselinedistances or separations between such imaging devices, disparities ofobjects within such images, and focal lengths of the respective imagingdevices.

Referring to FIG. 2, a block diagram of one system 200 for determiningstereo distance information using imaging devices integrated intopropeller blades in accordance with embodiments of the presentdisclosure is shown. The system 200 of FIG. 2 includes an aerial vehicle210 and a data processing system 270 connected to one another over anetwork 280, which may include the Internet, in whole or in part. Exceptwhere otherwise noted, reference numerals preceded by the number “2”shown in FIG. 2 indicate components or features that are similar tocomponents or features having reference numerals preceded by the number“1” shown in FIGS. 1A through 1E.

The aerial vehicle 210 includes a processor 212, a memory 214 and atransceiver 216. The aerial vehicle 210 further includes a controlsystem 220, a plurality of propulsion motors 230-1, 230-2 . . . 230-n, aplurality of propellers 240-1, 240-2 . . . 240-n and a plurality ofimaging devices 250-1, 250-2 . . . 250-n.

The processor 212 may be configured to perform any type or form ofcomputing function, including but not limited to the execution of one ormore machine learning algorithms or techniques. For example, theprocessor 212 may control any aspects of the operation of the aerialvehicle 210 and the one or more computer-based components thereon,including but not limited to the propulsion motors 230-1, 230-2 . . .230-n, the propellers 240-1, 240-2 . . . 240-n and the imaging devices250-1, 250-2 . . . 250-n. For example, the processor 212 may control theoperation of one or more control systems or modules, such as the controlsystem 220, for generating instructions for conducting operations of oneor more of the propulsion motors 230-1, 230-2 . . . 230-n, thepropellers 240-1, 240-2 . . . 240-n and the imaging devices 250-1, 250-2. . . 250-n. Such control systems or modules may be associated with oneor more other computing devices or machines, and may communicate withthe data processing system 270 or one or more other computer devices(not shown) over the network 280, through the sending and receiving ofdigital data.

The processor 212 may be a uniprocessor system including one processor,or a multiprocessor system including several processors (e.g., two,four, eight, or another suitable number), and may be capable ofexecuting instructions. For example, in some embodiments, the processor212 may be a general-purpose or embedded processor implementing any of anumber of instruction set architectures (ISAs), such as the x86,PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. Where theprocessor 212 is a multiprocessor system, each of the processors withinthe multiprocessor system may operate the same ISA, or different ISAs.

Additionally, the aerial vehicle 210 further includes one or more memoryor storage components 214 (such as databases or data stores) for storingany type of information or data, e.g., instructions for operating theaerial vehicle 210, or information or data captured during operations ofthe aerial vehicle 210. The memory 214 may be configured to storeexecutable instructions, flight paths, flight control parameters and/orother data items accessible by or to the processor 212. The memory 214may be implemented using any suitable memory technology, such as staticrandom access memory (SRAM), synchronous dynamic RAM (SDRAM),nonvolatile/Flash-type memory, or any other type of memory. In someembodiments, program instructions, flight paths, flight controlparameters and/or other data items may be received or sent via thetransceiver 216, e.g., by transmission media or signals, such aselectrical, electromagnetic, or digital signals, which may be conveyedvia a communication medium such as a wired and/or a wireless link.

The transceiver 216 may be configured to enable the aerial vehicle 210to communicate through one or more wired or wireless means, e.g., wiredtechnologies such as Universal Serial Bus (or “USB”) or fiber opticcable, or standard wireless protocols such as Bluetooth® or any WirelessFidelity (or “WiFi”) protocol, such as over the network 280 or directly.The transceiver 216 may further include or be in communication with oneor more input/output (or “I/O”) interfaces, network interfaces and/orinput/output devices, and may be configured to allow information or datato be exchanged between one or more of the components of the aerialvehicle 210, or to one or more other computer devices or systems (e.g.,other aerial vehicles, not shown) via the network 280. For example, insome embodiments, the transceiver 216 may be configured to coordinateI/O traffic between the processor 212 and one or more onboard orexternal computer devices or components. The transceiver 216 may performany necessary protocol, timing or other data transformations in order toconvert data signals from a first format suitable for use by onecomponent into a second format suitable for use by another component. Insome embodiments, the transceiver 216 may include support for devicesattached through various types of peripheral buses, e.g., variants ofthe Peripheral Component Interconnect (PCI) bus standard or theUniversal Serial Bus (USB) standard. In some other embodiments,functions of the transceiver 216 may be split into two or more separatecomponents, or integrated with the processor 212.

The control system 220 may include one or more electronic speedcontrols, power supplies, navigation systems and/or payload engagementcontrollers for controlling the operation of the aerial vehicle 210 andfor engaging with or releasing items, as desired. For example, thecontrol system 220 may be configured to cause or control the operationof one or more of the propulsion motors 230-1, 230-2 . . . 230-n, thepropellers 240-1, 240-2 . . . 240-n and the imaging devices 250-1, 250-2. . . 250-n, such as to cause one or more of the propulsion motors230-1, 230-2 . . . 230-n to rotate the propellers 240-1, 240-2 . . .240-n at a desired speed, in order to guide the aerial vehicle 210 alonga determined or desired flight path, and to cause one or more of theimaging devices 250-1, 250-2 . . . 250-n to capture any imaging data(e.g., still or moving images) as well as any associated audio dataand/or metadata. The control system 220 may further control otheraspects of the aerial vehicle 210, including but not limited to theoperation of one or more control surfaces (not shown) such as wings,rudders, ailerons, elevators, flaps, brakes, slats or other featureswithin desired ranges, or the enactment with or release of one or moreitems by one or more engagement systems (not shown). In someembodiments, the control system 220 may be integrated with one or moreof the processor 212, the memory 214 and/or the transceiver 216.

The propulsion motors 230-1, 230-2 . . . 230-n may be any type or formof motor (e.g., electric, gasoline-powered or any other type of motor)capable of generating sufficient rotational speeds of one or morepropellers or other components to provide lift and/or thrust forces tothe aerial vehicle 210 and any payload engaged thereby, to aeriallytransport the engaged payload thereby. For example, one or more of thepropulsion motors 230-1, 230-2 . . . 230-n may be a brushless directcurrent (DC) motor such as an outrunner brushless motor or an inrunnerbrushless motor.

The aerial vehicle 210 may include any number of such propulsion motors230-1, 230-2 . . . 230-n of any kind. For example, one or more of thepropulsion motors 230-1, 230-2 . . . 230-n may be aligned or configuredto provide forces of lift to the aerial vehicle 210, exclusively, whileone or more of the propulsion motors 230-1, 230-2 . . . 230-n may bealigned or configured to provide forces of thrust to the aerial vehicle210, exclusively. Alternatively, one or more of the propulsion motors230-1, 230-2 . . . 230-n may be aligned or configured to provide forcesof lift and forces of thrust to the aerial vehicle 210, as needed. Forexample, the propulsion motors 230-1, 230-2 . . . 230-n may be fixed intheir orientation on the aerial vehicle 210, or configured to vary theirrespective orientations, e.g., a tilt-rotor aircraft. Moreover, thepropulsion motors 230-1, 230-2 . . . 230-n may be aligned or configuredto operate with different capacities or ratings, or at different speeds,or coupled to propellers having different sizes and shapes.

The propellers 240-1, 240-2 . . . 240-n may be any rotors or rotatablesystems having a plurality of shaped blades joined to a hub or boss.Each of the propellers 240-1, 240-2 . . . 240-n is rotatably mounted toa mast or shaft associated with a respective one of the propulsionmotors 230-1, 230-2 . . . 230-n and configured to generate forces ofthrust when rotated within a fluid. Each of the propellers 240-1, 240-2. . . 240-n may include any number of blades, and may be fixed pitch,adjustable pitch or variable pitch in nature. Moreover, one or more ofthe propellers 240-1, 240-2 . . . 240-n may be banded or shielded in anymanner. In some embodiments, one or more of the propellers 240-1, 240-2. . . 240-n may be configured to rotate about a vertical axis, and toprovide forces of thrust in a vertical direction (e.g., upward)accordingly. In some other embodiments, one or more of the propellers240-1, 240-2 . . . 240-n may be configured to rotate about a horizontalaxis, and to provide forces of thrust in a horizontal direction (e.g.,forward) accordingly. In still other embodiments, one or more of thepropellers 240-1, 240-2 . . . 240-n may be configured to rotate aboutaxes that are neither horizontal nor vertical, and to provide forces ofthrust in directions corresponding to such axes accordingly.

The imaging devices 250-1, 250-2 . . . 250-n may be any form of opticalrecording devices that are embedded into surfaces of the respectivepropellers 240-1, 240-2 . . . 240-n and may be used to photograph orotherwise record imaging data of structures, facilities, terrain or anyother elements encountered during operation of the aerial vehicle 210,or for any other purpose. The imaging devices 250-1, 250-2 . . . 250-nmay include one or more sensors, memory or storage components andprocessors, and such sensors, memory components or processors mayfurther include one or more photosensitive surfaces, filters, chips,electrodes, clocks, boards, timers or any other relevant features (notshown). Such imaging devices 250-1, 250-2 . . . 250-n may captureimaging data in the form of one or more still or moving images of anykind or form, as well as any relevant audio signals or other informationduring the operation of the aerial vehicle 210, including but notlimited to when one or more of the propellers 240-1, 240-2 . . . 240-ninto which such imaging devices 250-1, 250-2 . . . 250-n are integratedis rotating at operational speeds.

The imaging devices 250-1, 250-2 . . . 250-n may communicate with theprocessor 212 and/or the control system 220, or with one another, by wayof a wired or wireless connection that may be dedicated or comprise allor part of an internal network (not shown). Additionally, the imagingdevices 250-1, 250-2 . . . 250-n may be adapted or otherwise configuredto communicate with the data processing system 270 by way of the network280. Although each of the propellers 240-1, 240-2 . . . 240-n of FIG. 2includes a single box corresponding to one of the imaging devices 250-1,250-2 . . . 250-n, those of ordinary skill in the pertinent arts willrecognize that any number or type of imaging devices may be provided inany number of the blades of the propellers 240-1, 240-2 . . . 240-n inaccordance with the present disclosure, including but not limited todigital cameras, depth sensors or range cameras, infrared cameras,radiographic cameras or other optical sensors.

In addition to the imaging devices 250-1, 250-2 . . . 250-n, the aerialvehicle 210 may also include any number of other sensors, components orother features for controlling or aiding in the operation of the aerialvehicle 210, including but not limited to one or more environmental oroperational sensors for determining one or more attributes of anenvironment in which the aerial vehicle 210 is operating, or may beexpected to operate, including extrinsic information or data orintrinsic information or data. For example, the aerial vehicle 210 mayinclude one or more Global Positioning System (“GPS”) receivers orsensors, compasses, speedometers, altimeters, thermometers, barometers,hygrometers, gyroscopes, air monitoring sensors (e.g., oxygen, ozone,hydrogen, carbon monoxide or carbon dioxide sensors), ozone monitors, pHsensors, magnetic anomaly detectors, metal detectors, radiation sensors(e.g., Geiger counters, neutron detectors, alpha detectors), attitudeindicators, depth gauges, accelerometers, or sound sensors (e.g.,microphones, piezoelectric sensors, vibration sensors or othertransducers for detecting and recording acoustic energy from one or moredirections).

The data processing system 270 includes one or more physical computerservers 272 having one or more computer processors 274 and any number ofdata stores 276 (e.g., databases) associated therewith, as well asprovided for any specific or general purpose. For example, the dataprocessing system 270 of FIG. 2 may be independently provided for theexclusive purpose of receiving, analyzing or storing imaging data orother information or data received from the aerial vehicle 210 or,alternatively, provided in connection with one or more physical orvirtual services configured to receive, analyze or store such imagingdata or other information or data, as well as one or more otherfunctions. The servers 272 may be connected to or otherwise communicatewith the processors 274 and the data stores 276, which may store anytype of information or data, including but not limited to acousticsignals, information or data relating to imaging data, or information ordata regarding environmental conditions, operational characteristics, orpositions, for any purpose. The servers 272 and/or the computerprocessors 274 may also connect to or otherwise communicate with thenetwork 280, as indicated by line 278, through the sending and receivingof digital data. For example, the data processing system 270 may includeany facilities, stations or locations having the ability or capacity toreceive and store information or data, such as media files, in one ormore data stores, e.g., media files received from the aerial vehicle210, or from one another, or from one or more other external computersystems (not shown) via the network 280. In some embodiments, the dataprocessing system 270 may be provided in a physical location. In othersuch embodiments, the data processing system 270 may be provided in oneor more alternate or virtual locations, e.g., in a “cloud”-basedenvironment. In still other embodiments, the data processing system 270may be provided onboard one or more aerial vehicles, including but notlimited to the aerial vehicle 210.

The network 280 may be any wired network, wireless network, orcombination thereof, and may comprise the Internet in whole or in part.In addition, the network 280 may be a personal area network, local areanetwork, wide area network, cable network, satellite network, cellulartelephone network, or combination thereof. The network 280 may also be apublicly accessible network of linked networks, possibly operated byvarious distinct parties, such as the Internet. In some embodiments, thenetwork 280 may be a private or semi-private network, such as acorporate or university intranet. The network 280 may include one ormore wireless networks, such as a Global System for MobileCommunications (GSM) network, a Code Division Multiple Access (CDMA)network, a Long Term Evolution (LTE) network, or some other type ofwireless network. Protocols and components for communicating via theInternet or any of the other aforementioned types of communicationnetworks are well known to those skilled in the art of computercommunications and thus, need not be described in more detail herein.

The computers, servers, devices and the like described herein have thenecessary electronics, software, memory, storage, databases, firmware,logic/state machines, microprocessors, communication links, displays orother visual or audio user interfaces, printing devices, and any otherinput/output interfaces to provide any of the functions or servicesdescribed herein and/or achieve the results described herein. Also,those of ordinary skill in the pertinent art will recognize that usersof such computers, servers, devices and the like may operate a keyboard,keypad, mouse, stylus, touch screen, or other device (not shown) ormethod to interact with the computers, servers, devices and the like, orto “select” an item, link, node, hub or any other aspect of the presentdisclosure.

The aerial vehicle 210 and/or the data processing system 270 may use anyweb-enabled or Internet applications or features, or any otherclient-server applications or features including E-mail or othermessaging techniques, to connect to the network 280, or to communicatewith one another, such as through short or multimedia messaging service(SMS or MMS) text messages. For example, the aerial vehicle 210 may beadapted to transmit information or data in the form of synchronous orasynchronous messages to the data processing system 270 or to any othercomputer device (e.g., to one or more other aerial vehicles) in realtime or in near-real time, or in one or more offline processes, via thenetwork 280. Those of ordinary skill in the pertinent art wouldrecognize that the aerial vehicle 210 or the data processing system 270may operate or be operated by any of a number of computing devices thatare capable of communicating over the network, including but not limitedto set-top boxes, personal digital assistants, digital media players,web pads, laptop computers, desktop computers, electronic book readers,and the like. The protocols and components for providing communicationbetween such devices are well known to those skilled in the art ofcomputer communications and need not be described in more detail herein.

The data and/or computer executable instructions, programs, firmware,software and the like (also referred to herein as “computer executable”components) described herein may be stored on a computer-readable mediumthat is within or accessible by computers or computer components such asthe processor 212 or the processor 274, or any other computers orcontrol systems utilized by the aerial vehicle 210 or the dataprocessing system 270 (e.g., by one or more other aerial vehicles), andhaving sequences of instructions which, when executed by a processor(e.g., a central processing unit, or “CPU”), cause the processor toperform all or a portion of the functions, services and/or methodsdescribed herein. Such computer executable instructions, programs,software, and the like may be loaded into the memory of one or morecomputers using a drive mechanism associated with the computer readablemedium, such as a floppy drive, CD-ROM drive, DVD-ROM drive, networkinterface, or the like, or via external connections.

Some embodiments of the systems and methods of the present disclosuremay also be provided as a computer-executable program product includinga non-transitory machine-readable storage medium having stored thereoninstructions (in compressed or uncompressed form) that may be used toprogram a computer (or other electronic device) to perform processes ormethods described herein. The machine-readable storage media of thepresent disclosure may include, but is not limited to, hard drives,floppy diskettes, optical disks, CD-ROMs, DVDs, ROMs, RAMs, erasableprogrammable ROMs (“EPROM”), electrically erasable programmable ROMs(“EEPROM”), flash memory, magnetic or optical cards, solid-state memorydevices, or other types of media/machine-readable medium that may besuitable for storing electronic instructions. Further, embodiments mayalso be provided as a computer executable program product that includesa transitory machine-readable signal (in compressed or uncompressedform). Examples of machine-readable signals, whether modulated using acarrier or not, may include, but are not limited to, signals that acomputer system or machine hosting or running a computer program can beconfigured to access, or including signals that may be downloadedthrough the Internet or other networks.

As is discussed above, an aerial vehicle may include an imaging devicethat is embedded or otherwise integrated within one or more blades of arotating propeller of an aerial vehicle. Images captured by the imagingdevice may be processed by stereo ranging algorithms or techniques todetermine ranges to any objects that are expressed in each of theimages. Referring to FIG. 3, a flow chart 300 of one process fordetermining stereo distance information using imaging devices integratedinto propeller blades in accordance with embodiments of the presentdisclosure is shown.

At box 310, an aerial vehicle having a digital camera embedded into asurface of a rotating propeller departs from an origin for transit to adestination. The aerial vehicle may be programmed to perform anymission, e.g., the delivery of a payload from the origin to thedestination, and the rotating propeller into which the digital camera isembedded may be provided for generating forces of thrust, forces oflift, or forces of thrust and lift.

At box 320, the aerial vehicle captures a first image using the digitalcamera with the propeller at an angle of orientation θ₁ at time t₁. Thedigital camera may be programmed to begin capturing one or more imagesautomatically, upon an arrival of the aerial vehicle at a givenlocation, upon the aerial vehicle reaching a given speed or a givenaltitude, upon sensing one or more objects (e.g., collision risks)nearby, upon detecting any predetermined environmental or operatingcondition, or for any other reason. At box 330, the aerial vehiclecaptures a second image using the digital camera with the propeller atan angle of orientation θ₂ at time t₂. The digital camera may beprogrammed to capture another image based on an angular orientation orposition of the propeller, at a predetermined time, or for any otherreason. For example, in some embodiments, the digital camera may beconfigured to capture the second image when the angle of orientation ofthe propeller is one hundred eighty degrees (180°) greater or less thanthe angle of orientation of the propeller when the first image wascaptured. As yet another example, in some embodiments, the digitalcamera may be configured to capture the second image after apredetermined elapsed time following the capture of the first image.

At box 340, the first image and the second image are oriented withrespect to one another based on the difference Δθ in the angles oforientation θ₂ and θ₁, or θ₂−θ₁. For example, the first image may bereoriented with respect to the second image, or the second image may bereoriented with respect to the first image. Alternatively, each of thefirst image and the second image may be independently oriented withrespect to a common standard angle.

At box 350, the first image and the second image are subjected to acontent-based analysis. For example, each of the first image and thesecond image may be evaluated to identify attributes of any pointsrepresented in either or both of the first image and the second image,including but not limited to any number of edges, contours, outlines,colors, textures, silhouettes, shapes or other characteristics ofobjects, or portions of objects, expressed therein using one or morealgorithms or machine-learning tools. Some such algorithms or tools mayinclude, but are not limited to, Canny edge detectors or algorithms;Sobel operators, algorithms or filters; Kayyali operators; Roberts edgedetection algorithms; Prewitt operators; Frei-Chen methods; or any otheralgorithms or techniques that may be known to those of ordinary skill inthe pertinent arts.

At box 360, an object is identified in each of the first image and thesecond image. For example, an object may be identified in one of theimages, and a search may be conducted for the object in another of theimages. In some embodiments, the epipolar lines of the respective imagesmay be rectified using one or more transformations, in order to alignthe epipolar lines with scan lines of the images, thereby facilitating asearch for an object that was identified in one of the images in theother of the images. Thereafter, pixels corresponding to points of theobject in one image may be identified in the other of the images, e.g.,by matching pixels between the respective images, until the object isidentified.

At box 370, range data to the object is determined based on a disparityof the object within the first image and the second image, a baselineseparation of the digital camera at time t₁ and time t₂, and a focallength of the digital camera according to one or more stereo algorithmsand/or techniques. As is noted above, the disparity is defined as theseparation of a given point between two images of the same scene, whilethe baseline separation is a distance between positions of the digitalcamera when the first image was captured (e.g., at time t₁) and a secondposition of the digital camera when the second image was captured (e.g.,at time t₂), and the focal length of the digital camera is a distancebetween a sensor and a lens within the digital camera. Stereo rangingalgorithms or techniques may use the disparity, the baseline separationand the focal length in order to determine a range or a distance to theobject, or ranges or distances to one or more aspects of the object. Atbox 380, the range data is stored in one or more data stores, and theprocess ends. The range data may be utilized for any purpose, e.g., fornavigation, guidance, surveillance, collision avoidance, or any otherpurpose.

Referring to FIGS. 4A, 4B and 4C, views of aspects of one system 400 fordetermining stereo distance information using imaging devices integratedinto a propeller blade in accordance with embodiments of the presentdisclosure are shown. Except where otherwise noted, reference numeralspreceded by the number “4” shown in FIG. 4A, FIG. 4B or FIG. 4C indicatecomponents or features that are similar to components or features havingreference numerals preceded by the number “2” shown in FIG. 2 or by thenumber “1” shown in FIGS. 1A through 1E.

As is shown in FIG. 4A, an aerial vehicle 410 having a propeller 440with an imaging device 450 embedded in an underside thereof is shown.The aerial vehicle 410 is shown as approaching a point P(x,y,z) in spacewithin a scene having one or more structures, obstacles and/or otherobjects (e.g., a home and a bicycle). The point P(x,y,z) is within afield of view of the imaging device 450 at times t₁ and t₂, when thepropeller 440 is aligned at angles θ₁ and θ₂. As is also shown in FIG.4A, an image 40-1 captured at time t₁, and with the propeller 440aligned at the angle θ₁ includes a projection U₁ of the point P(x,y,z).An inverted image 40-2′ captured at time t₂ and with the propeller 440aligned at the angle θ₂ includes a projection U₂ of the point P(x,y,z).Where the angle θ₂ and the angle θ₂ are one hundred eighty degrees(180°) apart, the inverted image 40-2′ may be derived by rotating animage captured at time t₂ by one hundred eighty degrees (180°). Thebaseline separation of the position of the imaging device 450 at time t₁and the position of the imaging device 450 at time t₂ is approximately2r, or twice the radius r of the distance from a hub of the propeller440 to the imaging device 450.

A range or distance to the point P(x,y,z) in space, or to one or moreother points expressed within both the image 40-1 and the inverted image40-2′, may be determined by virtually overlapping the images 40-1, 40-2′and determining disparities between projections of each point that isshown in both of the image 40-1, 40-2′. As is shown in FIG. 4B, wherethe image 40-1 and the inverted image 40-2′ are overlapped upon oneanother, a disparity between the projections U₁, U₂ of the pointP(x,y,z) in space within the images 40-1, 40-2′ is apparent.

As is shown in FIG. 4C, the range z to the point P(x,y,z) may bedetermined by stereo ranging using the known baseline separation 2r, thefocal length f, the projections U₁, U₂ of the point P(x,y,z) within theimage 40-1 and the rotated image 40-2′. Stereo ranging algorithms andtechniques may automatically determine the position of the pointP(x,y,z) and, therefore, the range z to the point P(x,y,z), bytriangulating the positions of the imaging device 450 at time t₁ andtime t₂, the projections U₁, U₂ using the focal length f and thebaseline separation 2r.

Although FIGS. 4A, 4B and 4C depict the determination of a position ofand/or range to a single point P(x,y,z) in space, the systems andmethods of the present disclosure are not so limited, and may be used todetermine positions of and/or ranges to each of the points that appearsin both of the images 40-1, 40-2′, according to the same stereo rangingalgorithms or techniques, or to one or more other algorithms ortechniques. Using such positions or ranges, any form ofthree-dimensional representation of the scene and/or the structures,objects or other features may be constructed, including but not limitedto a point cloud representing pixel-level positions of each of thepoints appearing in both of the images 40-1, 40-2′, or a depth map,e.g., the depth map 15 of FIG. 1E, that shows average or nominal rangesto one or more of the objects within the scene, along with one or moretolerances or confidence levels representative of the accuracy orprecision of such ranges. Because the propeller 440 of the aerialvehicle 410 of FIG. 4A typically rotates at angular velocities ofseveral thousand revolutions per minute, the imaging device 450effectively appears in two places simultaneously (e.g., within smallfractions of seconds). Images captured by the single imaging device 450may, therefore, be utilized in determining ranges from the propeller 440to one or more objects expressed in two or more of such images.

As is noted above, images captured by an imaging device integrated intoa propeller surface may be oriented with respect to each other on anybasis, such as by reorienting one image with respect to another image,or by reorienting each of the images with respect to a common standard.Referring to FIGS. 5A through 5C, views of aspects of one system fordetermining stereo distance information using imaging devices integratedinto propeller blades in accordance with embodiments of the presentdisclosure are shown. Except where otherwise noted, reference numeralspreceded by the number “5” shown in FIG. 5A, FIG. 5B or FIG. 5C indicatecomponents or features that are similar to components or features havingreference numerals preceded by the number “4” shown in FIG. 4A, FIG. 4Bor FIG. 4C, by the number “2” shown in FIG. 2 or by the number “1” shownin FIGS. 1A through 1E.

Referring to FIG. 5A, two images 50A-1, 50A-2 captured by an imagingdevice embedded in a propeller are shown. The image 50A-1 was capturedwith the propeller at an angle θ₁ of zero degrees (0°), while the image50A-2 was captured with the propeller at an angle θ₂ that is greaterthan zero degrees. Therefore, in order to properly orient the images50A-1, 50A-2 with respect to one another, the image 50A-2 may be rotatedby the angle θ₂ in an opposite direction to form an image 50A-2′,thereby ensuring that the images 50A-1, 50A-2′ are properly aligned withrespect to one another, and canceling out the effects of the propeller'sorientation on the respective images 50A-1, 50A-2. With the images50A-1, 50A-2′ properly aligned with respect to one another, pointscorresponding to objects that appear in each of the images 50A-1, 50A-2′may be identified, and distances (or ranges) to such objects may bedetermined, e.g., by stereo ranging algorithms or techniques.

Conversely, referring to FIG. 5B, two images 50B-1, 50B-2 captured by animaging device embedded in a propeller are shown. The image 50B-1 wascaptured with the propeller at an angle θ₁ that is less than zerodegrees (0°), while the image 50B-2 was captured with the propeller atan angle θ₂ that is equal to zero degrees. Therefore, in order toproperly orient the images 50B-1, 50B-2 with respect to one another, theimage 50B-1 may be rotated by the angle θ₁ in an opposite direction toform an image 50B-1′, thereby ensuring that the images 50B-1′, 50B-2 areproperly aligned with respect to one another, and canceling out theeffects of the propeller's orientation on the respective images 50B-1,50B-2.

Images may also be reoriented with respect to standard angle oforientation, rather than an angle of orientation of either of theimages. Referring to FIG. 5C, two images 50C-1, 50C-2 captured by animaging device embedded in a propeller are shown. The image 50C-1 wascaptured with the propeller at an angle θ₁ that is greater than zerodegrees (0°), while the image 50C-2 was captured with the propeller atan angle θ₂ that is less than zero degrees. Therefore, in order toproperly orient the images 50C-1, 50C-2 with respect to one another, ata standard angle of orientation θ_(STD), the image 50C-1 may be rotatedby an angle (θ₁−θ_(STD)) in an opposite direction to form an image50C-1′, and the image 50C-2 may be rotated by an angle (θ_(STD)−θ₂) inan opposite direction to form an image 50C-2′, thereby ensuring that theimages 50C-1′, 50C-2 are properly aligned with respect to one another,at the standard angle of orientation θ_(STD), and canceling out theeffects of the propeller's orientation on the respective images 50C-1,50C-2.

As is discussed above, the systems and methods of the present disclosuremay be used to determine positions of any number of points (or ranges ordistances to such points) appearing in two or more images of a scenethat are captured by an imaging device integrated into a blade of arotating propeller of an aerial vehicle, or by any number of otherimaging devices provided on the aerial vehicle. Using such positions (orranges or distances thereto), a three-dimensional representation of ascene may be constructed, including a point cloud, a depth map, or anyother virtual structure representing the geographic or topographicallayout of the scene. Referring to FIG. 6, a flow chart 600 of oneprocess for determining stereo distance information using imagingdevices integrated into propeller blades in accordance with embodimentsof the present disclosure is shown. At box 610, an aerial vehicleoperates with at least one imaging device embedded into at least onepropeller surface. The aerial vehicle may include a single imagingdevice embedded into a single blade of one propeller, such as theimaging device 150 embedded into the first blade 144-4 of the propeller140-4 of FIGS. 1A through 1D, or multiple imaging devices that areembedded into surfaces of multiple blades of multiple propellers.

At box 620, the aerial vehicle begins a ranging operation, in which theaerial vehicle is configured to determine ranges or distances to anynumber of points (e.g., points corresponding to surfaces of structuresor objects) that are present below the aerial vehicle, such as whensearching for a suitable location for the aerial vehicle to land for anyreason. Alternatively, the aerial vehicle may conduct ranging operationsto determine ranges or distances to points that are above, forward of,behind, to the left of or to the right of the aerial vehicle, or in anyother direction with respect to the aerial vehicle, and appearing in twoor more images captured by an imaging device integrated into anoperating propeller.

At box 630, the imaging device captures an image with the propellerblade at a first selected angular orientation. The blade may be alignedat any angle with respect to the aerial vehicle, e.g., transverse to adirection of travel, along the direction of travel, or in any otherorientation. At box 635, the imaging device stores the first image in anonboard memory, e.g., in one or more databases, data stores or othercomponents provided aboard the aerial vehicle. Alternatively, in someembodiments, the aerial vehicle may transmit the first image to aground-based or “cloud”-based processing facility, or to one or more ofsuch facilities, using one or more transceivers. The first image may betransmitted to one or more of such facilities in a synchronous or anasynchronous process, e.g., in real time or in near-real time, andeither singly or as part of a batch process. Alternatively, the firstimage, and any number of other images, may be transmitted to anotherfacility upon a completion of a mission, e.g., for a forensic analysisof any surface features expressed in images captured by the imagingdevice.

At box 640, the imaging device recognizes a plurality of points withinthe first image. For example, such points may correspond to one or moreedges, contours, outlines, colors, textures, silhouettes, shapes orother characteristics of objects shown in the first image, or portionsof such objects, that may be identified therein using one or morealgorithms or machine-learning tools. Alternatively, the points may berecognized by computer devices provided at a ground-based or“cloud”-based processing facility, or at one or more of such facilities,e.g., in real time, in near-real time, or at any later time.

At box 650, the imaging device captures a next image with the propellerblade in a next selected angular orientation. For example, the imagingdevice may be configured to capture the next image at a predeterminedtime, or when the propeller blade reaches a predetermined orientation.In some embodiments, the predetermined time or the predeterminedorientation may be selected based on an operating speed (e.g., anangular velocity) of the propeller or of a motor to which the propelleris rotatably coupled. At box 655, the imaging device stores the nextimage in an onboard memory, and at box 660, the imaging devicerecognizes at least some of the plurality of points that were previouslyrecognized in the next image. Alternatively, as is discussed above, theimaging device may transmit the next image to a ground-based or“cloud”-based processing facility, or to one or more of such facilities,for storage or processing, e.g., in real time, in near-real time, or atany later time.

At box 670, the imaging device determines ranges to the recognizedpoints based on disparities of such points within the most recentimages, baseline separations of the imaging device when each of the mostrecent images was captured and/or a focal length of the imaging devicewhen the most recent images were captured. For example, as is discussedabove, the images may be realigned with respect to one another (e.g.,with respect to the contents thereof), and projections of the pointsthat are recognized within each of the two most recent images may beidentified therein. Disparities between such projections may then bedetermined, and using the disparities, the baseline separation and afocal length of the imaging device, ranges to the recognized points maybe determined according to one or more stereo ranging algorithms ortechniques. At box 680, ranges to the recognized points at the time ofthe most recent image are stored in the onboard memory or,alternatively, on one or more ground-based or “cloud”-based facilities,and may be used for any purpose. For example, the ranges may be used todefine a point cloud or depth map of a given region, and the point cloudor depth map may be utilized to identify a specific location for theaerial vehicle to land, or for any other reason for which the rangingoperation was begun at box 620.

At box 690, whether the ranging operation is complete is determined. Forexample, if the ranging objectives of the operation have been completed(e.g., the identification of a landing site or location, or theconstruction of a point cloud or depth map with suitable resolution orprecision), then the process ends. If the ranging operation is notcomplete, however, then the process returns to box 650, where theimaging device captures a next image with the propeller blade in a nextselected angular orientation, e.g., at a predetermined time, or when thepropeller blade reaches the selected angular orientation, and theprocess repeats itself until the ranging operation is determined to havebeen completed. Given the high angular velocities typically observedduring the operation of a propeller aboard an aerial vehicle (e.g., aUAV), the accuracy of such a point cloud or depth map may be rapidly andprecisely refined based on pairs of images captured at high frequencies.

As is discussed above, the systems and methods of the present disclosuremay be utilized in conducting any ranging operations using an aerialvehicle, including but not limited to the construction of depth maps ofa given region. Referring to FIG. 7, a view of aspects of one system 700for determining stereo distance information using imaging devicesintegrated into a propeller blade in accordance with embodiments of thepresent disclosure is shown. Except where otherwise noted, referencenumerals preceded by the number “7” shown in FIG. 7 indicate componentsor features that are similar to components or features having referencenumerals preceded by the number “5” shown in FIG. 5A, FIG. 5B or FIG.5C, by the number “4” shown in FIG. 4A, FIG. 4B or FIG. 4C, by thenumber “2” shown in FIG. 2 or by the number “1” shown in FIGS. 1Athrough 1E.

As is shown in FIG. 7, a propeller 740 having a first blade 744 and asecond blade 746 mounted about a hub 742 is shown. The first blade 744includes an imaging device 750 embedded in one surface thereof. Thepropeller 740 is configured to rotate about the hub 742 under power,e.g., by one or more motors joined to the hub 742 (not shown).

The imaging device 750 may be configured to capture any number of imagesat regular angular intervals, and to utilize such images in definingdepth maps or other representations of depth or range information. As isshown in FIG. 7, the imaging device 750 captures images 70-1, 70-2,70-3, 70-4, and so on and so forth, with the first blade 744 aligned atangles of θ and −θ, respectively, with respect to the hub 742. Theimages 70-1, 70-2, 70-3, 70-4 may then be analyzed to recognize one ormore objects therein, and processed to determine ranges to such objects,or ranges to regions of the images corresponding to such objects,according to one or more stereo ranging algorithms or techniques. Theranges determined based on analyses of the images 70-1, 70-2, 70-3, 70-4may be aggregated into a plurality of depth maps 75-1, 75-2, 75-3, andso on and so forth, which may be iteratively updated using successivelycaptured images in order to determine whether ranges to such objects orregions have changed, and to what extent, thereby effectivelyconstructing a dynamic point cloud, depth map or other three dimensionalmodel of a scene in which the aerial vehicle is operating that changesover time. The images 70-1, 70-2, 70-3, 70-4 may be rotated or otherwiserealigned, as necessary, prior to analyzing such images with respect toone another, or prior to searching for the one or more objects therein.

Therefore, in accordance with the present disclosure, dynamic range mapsmay be created and updated, as necessary, when ranging information isdetermined using images captured using imaging devices embedded orotherwise integrated into a surface of a blade of a rotating propeller.Where a propeller is rotating at a sufficiently high angular velocity,e.g., on the order of thousands of revolutions per minute (rpm), animaging device embedded in a blade of the propeller may effectively bepresent in two locations at once, and images captured by the imagingdevice in the different locations may be analyzed according to one ormore stereo ranging algorithms or techniques in order to derive rangeinformation thereof. The ranges may be determined for any reason,including but not limited to identifying a site or location for landingthe aerial vehicle (e.g., a sufficiently large, flat and durable surfacethat may accommodate one or more dimensions of the aerial vehicle),navigating the aerial vehicle (e.g., to identify terrain or contours fornavigational purposes), or searching for or avoiding one or moreairborne or ground-based objects.

In accordance with the present disclosure, any number of propellerblades may include any number of imaging devices integrated therein, andsuch imaging devices may be integrated at different radii with respectto hubs to which the propeller blades are mounted. Referring to FIGS. 8Aand 8B, views of propeller blades 840A, 840B having imaging devicesintegrated therein for determining stereo distance information inaccordance with embodiments of the present disclosure are shown. Exceptwhere otherwise noted, reference numerals preceded by the number “8”shown in FIG. 8A or FIG. 8B indicate components or features that aresimilar to components or features having reference numerals preceded bythe number “7” shown in FIG. 7, by the number “5” shown in FIG. 5A, FIG.5B or FIG. 5C, by the number “4” shown in FIG. 4A, FIG. 4B or FIG. 4C,by the number “2” shown in FIG. 2 or by the number “1” shown in FIGS. 1Athrough 1E.

As is shown in FIG. 8A, the propeller 840A includes a hub 842A and apair of blades 844A-1, 844A-2. The blade 844A-1 includes an imagingdevice 850A-1 embedded therein at a distance r_(A-1) from the hub 842A.The blade 844A-2 includes an imaging device 850A-2 embedded therein at adistance r_(A-2) from the hub 842A.

The propeller 840A may be utilized to capture depth information in anymode or format. For example, because the imaging devices 850A-1, 850A-2are positioned at a fixed baseline distance from one another, e.g., asum of the distances r_(A-1), r_(A-2), the imaging devices 850A-1,850A-2 may capture images in concert with one another, and such imagesmay be evaluated to determine range information therefrom, e.g.,according to one or more stereo ranging algorithms or techniques.Alternatively, the imaging devices 850A-1, 850A-2 may be used toindependently capture images that may be analyzed and processed in orderto determine range information to any number of points (e.g., pointscorresponding to surfaces of one or more objects). For example, theimaging device 850A-1 may be configured to capture images when the blade844A-1 is aligned at one or more predetermined orientations or at one ormore predetermined times, and the imaging device 850A-2 may beseparately configured to capture images when the blade 844A-2 is alignedat one or more predetermined orientations or at one or morepredetermined times. Images captured separately by the respectiveimaging devices 850A-1, 850A-2 may be used to determine depthinformation regarding ranges from the respective blades 844A-1, 844A-2to one or more points. The distances r_(A-1), r_(A-2) need not be equalto one another, and each of the imaging devices 850A-1, 850A-2 may havedifferent capacities, specifications or ratings integrated therein, withsuch imaging devices being utilized for different purposes accordingly.

As is shown in FIG. 8B, the propeller 840B includes a hub 8423B havingthree blades 844B-1, 844B-2, 844B-3. The blade 844B-1 includes animaging device 850B-1 embedded therein at a distance r_(B-1) from thehub 842B. The blade 844B-2 includes an imaging device 850B-2 embeddedtherein at a distance r_(B-2) from the hub 842B. The blade 844B-3includes an imaging device 850B-3 embedded therein at a distance r_(B-3)from the hub 842B. As is discussed above with regard to the propeller840A and the imaging devices 850A-1, 850A-2 of FIG. 8A, the imagingdevices 850B-1, 850B-2, 850B-3 may capture images in concert with oneanother, and such images may be evaluated to determine range informationtherefrom, e.g., according to one or more stereo ranging algorithms ortechniques. A baseline distance between any two of the imaging devices850B-1, 850B-2, 850B-3 may be determined according to triangleproperties and/or traditional trigonometric functions. Alternatively,the imaging devices 850B-1, 850B-2, 850B-3 may be used to independentlycapture images, e.g., when the blades 844B-1, 844B-2, 844B-3 are alignedat predetermined orientations or at predetermined times, and such imagesmay be analyzed and processed in order to determine range information toone or more points in space. The distances r_(B-1), r_(B-2), r_(B-3)need not be equal to one another. Each of the imaging devices 850B-1,850B-2, 850B-3 may have different capacities, specifications or ratingsintegrated therein, with such imaging devices being utilized fordifferent purposes accordingly.

In accordance with the present disclosure, a determination of depthinformation using images captured by imaging devices integrated intopropeller blades aboard aerial vehicles may be augmented by imagescaptured using imaging devices that are integrated into non-rotatingportions of an aerial vehicle, such as a frame, a fuselage, a controlsurface or one or more other components or surfaces thereof. Referringto FIGS. 9A through 9D, views of aspects of one system for determiningstereo distance information using imaging devices integrated intopropeller blades in accordance with embodiments of the presentdisclosure are shown. Except where otherwise noted, reference numeralspreceded by the number “9” shown in FIG. 9A, FIG. 9B, FIG. 9C or FIG. 9Dindicate components or features that are similar to components orfeatures having reference numerals preceded by the number “8” shown inFIG. 8A or FIG. 8B, by the number “7” shown in FIG. 7, by the number “5”shown in FIG. 5A, FIG. 5B or FIG. 5C, by the number “4” shown in FIG.4A, FIG. 4B or FIG. 4C, by the number “2” shown in FIG. 2 or by thenumber “1” shown in FIGS. 1A through 1E.

Referring to FIG. 9A, an aerial vehicle 910 includes a control center920, a plurality of motors 930-1, 930-2, 930-3, 930-4 and a plurality ofpropellers 940-1, 940-2, 940-3, 940-4, with each of the propellers940-1, 940-2, 940-3, 940-4 rotatably coupled to one of the motors 930-1,930-2, 930-3, 930-4. The aerial vehicle 910 further includes a pluralityof imaging devices 950-1, 950-2, 950-3, 950-4, 950-5, with each of theimaging devices 950-1, 950-2, 950-3, 950-4 being mounted to a blade ofone of the propellers 940-1, 940-2, 940-3, 940-4, and with the imagingdevice 950-5 being mounted to a frame of the aerial vehicle 910, e.g.,beneath the control center 920.

Images captured by the non-rotating imaging device 950-5 may be used inconjunction with images captured by one or more of the rotating imagingdevices 950-1, 950-2, 950-3, 950-4 to determine information regardingdistances or ranges to points beneath the aerial vehicle 910. Forexample, as is shown in FIG. 9B, the propeller 940-4 includes a firstblade 944-4 and a second blade 946-4 mounted to a hub 942-4 joined tothe motor 930-4, with the first blade 944-4 having the imaging device950-4 embedded therein at a distance r from the hub 942-4. The motor930-4 and the hub 942-4 are located at a distance 1 from thenon-rotating imaging device 950-5. Thus, a baseline separation betweenthe non-rotating imaging device 950-5 and the rotating imaging device950-4 when the first blade 944-4 is oriented at the angle θ₁ shown inFIG. 9B is equal to l+r. As is shown in FIG. 9C, a baseline separationbetween the non-rotating imaging device 950-5 and the rotating imagingdevice 950-4 when the first blade 944-4 is oriented at the angle θ₂shown in FIG. 9C is equal to l−r. Any number of other baselineseparations may be determined according to triangle properties and/ortraditional trigonometric functions based on the angle of orientation ofthe first blade 944-4.

Therefore, images captured using the non-rotating imaging device 950-5and the rotating imaging device 950-4, or any of the other rotatingimaging devices 950-1, 950-2, 950-3, may be used in making independentdeterminations of depth information to one or more points beneath theaerial vehicle 910, e.g., by stereo ranging algorithms or techniques.Such determinations may increase the accuracy of the available depthinformation associated with the aerial vehicle 910, or any depth maps ordepth models generated therefrom, and may filter out outlying orinaccurate results determined by stereo ranging algorithms or techniquesusing images captured by any one of the imaging devices 950-1, 950-2,950-3, 950-4, 950-5.

Moreover, any of the imaging devices 950-1, 950-2, 950-3, 950-4, 950-5may be configured to capture imaging data simultaneously, and any ofsuch imaging data may be processed for any purpose, including but notlimited to determining stereo distance information to one or more pointsbeneath the aerial vehicle 910. As is shown in FIG. 9D, the aerialvehicle 910 and/or the control center 920 may operate the imagingdevices 950-1, 950-2, 950-3, 950-4, 950-5 separately or in concert tocapture still or moving images, and any associated audio information ormetadata, regarding any number of points beneath the aerial vehicle 910that appear in two or more images captured by one or more of the imagingdevices 950-1, 950-2, 950-3, 950-4, 950-5.

A propeller having an imaging device integrated into a blade, or a motorfor rotating such a propeller about an axis, may further include animaging device aligned along the axis, and images captured by therespective imaging devices may be used to determine ranges or distancesto points according to one or more stereo ranging algorithms ortechniques. Referring to FIGS. 10A through 10C, views of aspects of onesystem for determining stereo distance information using imaging devicesintegrated into propeller blades in accordance with embodiments of thepresent disclosure are shown. Except where otherwise noted, referencenumerals preceded by the number “10” shown in FIG. 10A, FIG. 10B or FIG.10C indicate components or features that are similar to components orfeatures having reference numerals preceded by the number “9” shown inFIG. 9A, FIG. 9B, FIG. 9C or FIG. 9D, by the number “8” shown in FIG. 8Aor FIG. 8B, by the number “7” shown in FIG. 7, by the number “5” shownin FIG. 5A, FIG. 5B or FIG. 5C, by the number “4” shown in FIG. 4A, FIG.4B or FIG. 4C, by the number “2” shown in FIG. 2 or by the number “1”shown in FIGS. 1A through 1E.

As is shown in FIGS. 10A through 10C, a motor 1030 includes a propeller1040 rotatably coupled thereto. The propeller 1040 includes a firstblade 1044 and a second blade 1046, and an imaging device 1050-1 isintegrated into an underside of the first blade 1044. Additionally, animaging device 1050-2 is also integrated into an underside of the motor1030. The imaging device 1050-1 and the imaging device 1050-2 arealigned such that their respective axes of orientation are parallel toone another, and the fields of view of the respective imaging devices1050-1, 1050-2 intersect and overlap to a significant extent, with theintersection and overlapping beginning at a nominal distance below themotor 1030.

The motor 1030 is configured to rotate the propeller 1040 about an axisthat coincides with the axis of orientation of the imaging device1050-2. Therefore, because the imaging device 1050-1 is integrated intoan underside of the first blade 1044 at a distance r from a hub or bossof the propeller 1040, the imaging device 1050-1 and the imaging device1050-2 remain at the fixed distance r from one another, regardless of anangle of orientation of the first blade 1044. For example, as is shownin FIG. 10A, FIG. 10B and FIG. 10C, whether the first blade 1044 isaligned at a first angle θ₁, a second angle θ₂, or a third angle θ₃.Thus, the propeller 1040 and the motor 1030, and the imaging devices1050-1, 1050-2 provided thereon, may be utilized to determine stereodistance information to points within fields of view of both of theimaging devices 1050-1, 1050-2 while the first blade 1044 is at anyangular orientation, e.g., at one or more of the first angle θ₁, thesecond angle θ₂, or the third angle θ₃, or at any intervening angle.Furthermore, because the imaging device 1050-2 may be used tocontinuously or substantially continuously capture images of a scene athigh rates of speed, the imaging device 1050-2 may capture images of thescene from different perspectives, thereby enhancing the accuracy of anystereo distance information determined using such images, and improvingthe resolution of any point clouds, depth models or otherrepresentations generated from the stereo distance information.

Although the disclosure has been described herein using exemplarytechniques, components, and/or processes for implementing the systemsand methods of the present disclosure, it should be understood by thoseskilled in the art that other techniques, components, and/or processesor other combinations and sequences of the techniques, components,and/or processes described herein may be used or performed that achievethe same function(s) and/or result(s) described herein and which areincluded within the scope of the present disclosure.

As used herein, the terms “forward” flight or “horizontal” flight referto flight in a direction substantially parallel to the ground (i.e., sealevel). As used herein, the term “vertical” flight refers to flight in adirection extending substantially radially outward from a center of theEarth. Those of ordinary skill in the pertinent arts will recognize thatflight trajectories may include components of both “forward” flight or“horizontal” flight and “vertical” flight vectors.

Although some of the embodiments disclosed herein reference the use ofunmanned aerial vehicles to deliver payloads from warehouses or otherlike facilities to customers, those of ordinary skill in the pertinentarts will recognize that the systems and methods disclosed herein arenot so limited, and may be utilized in connection with any type or formof aerial vehicle (e.g., manned or unmanned) having fixed or rotatingwings for any intended industrial, commercial, recreational or otheruse.

It should be understood that, unless otherwise explicitly or implicitlyindicated herein, any of the features, characteristics, alternatives ormodifications described regarding a particular embodiment herein mayalso be applied, used, or incorporated with any other embodimentdescribed herein, and that the drawings and detailed description of thepresent disclosure are intended to cover all modifications, equivalentsand alternatives to the various embodiments as defined by the appendedclaims. Moreover, with respect to the one or more methods or processesof the present disclosure described herein, including but not limited tothe processes represented in the flow charts of FIG. 3 or 6, orders inwhich such methods or processes are presented are not intended to beconstrued as any limitation on the claimed inventions, and any number ofthe method or process steps or boxes described herein can be combined inany order and/or in parallel to implement the methods or processesdescribed herein. Also, the drawings herein are not drawn to scale.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey in apermissive manner that certain embodiments could include, or have thepotential to include, but do not mandate or require, certain features,elements and/or steps. In a similar manner, terms such as “include,”“including” and “includes” are generally intended to mean “including,but not limited to.” Thus, such conditional language is not generallyintended to imply that features, elements and/or steps are in any wayrequired for one or more embodiments or that one or more embodimentsnecessarily include logic for deciding, with or without user input orprompting, whether these features, elements and/or steps are included orare to be performed in any particular embodiment.

Disjunctive language such as the phrase “at least one of X, Y, or Z,” or“at least one of X, Y and Z,” unless specifically stated otherwise, isotherwise understood with the context as used in general to present thatan item, term, etc., may be either X, Y, or Z, or any combinationthereof (e.g., X, Y, and/or Z). Thus, such disjunctive language is notgenerally intended to, and should not, imply that certain embodimentsrequire at least one of X, at least one of Y, or at least one of Z toeach be present.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

Language of degree used herein, such as the terms “about,”“approximately,” “generally,” “nearly” or “substantially” as usedherein, represent a value, amount, or characteristic close to the statedvalue, amount, or characteristic that still performs a desired functionor achieves a desired result. For example, the terms “about,”“approximately,” “generally,” “nearly” or “substantially” may refer toan amount that is within less than 10% of, within less than 5% of,within less than 1% of, within less than 0.1% of, and within less than0.01% of the stated amount.

Although the invention has been described and illustrated with respectto illustrative embodiments thereof, the foregoing and various otheradditions and omissions may be made therein and thereto withoutdeparting from the spirit and scope of the present disclosure.

What is claimed is:
 1. An aerial vehicle comprising: a frame; apropulsion motor mounted to the frame, wherein the propulsion motor isconfigured to rotate a shaft about an axis defined by the shaft; apropeller having a plurality of blades, wherein the propeller isrotatably coupled to the shaft; a digital camera embedded in anunderside of one of the plurality of blades; and at least one computerprocessor, wherein the at least one computer processor is configured toat least: cause the propulsion motor to rotate the propeller about theaxis at a predetermined speed; cause the digital camera to capture afirst digital image at a first time, wherein the one of the plurality ofblades having the digital camera embedded therein is aligned in a firstangular orientation at the first time; cause the digital camera tocapture a second digital image at a second time, wherein the one of theplurality of blades having the digital camera embedded therein isaligned in a second angular orientation at the second time; determine adistance between a first position of the digital camera at the firsttime and a second position of the digital camera at the second time; anddetermine an altitude of the aerial vehicle above at least a portion ofa surface below the aerial vehicle based at least in part on thedistance, a focal length of the digital camera, the first digital imageand the second digital image.
 2. The aerial vehicle of claim 1, whereinthe at least one computer processor is further configured to at least:recognize a first representation of the portion of the surface below theaerial vehicle within at least a portion of the first image; andrecognize a second representation of the portion of the surface belowthe aerial vehicle within at least a portion of the second image,wherein the altitude of the aerial vehicle is determined based at leastin part on the distance, the focal length, the first representation andthe second representation.
 3. The aerial vehicle of claim 2, wherein theat least one computer processor is further configured to at least:define a first line extending from the first position of the imagingdevice through the first representation; define a second line extendingfrom the second position of the imaging device through the secondrepresentation; and identify an intersection of the first line and thesecond line, wherein the altitude of the aerial vehicle is determinedbased at least in part on the intersection of the first line and thesecond line.
 4. The aerial vehicle of claim 1, wherein the at least onecomputer processor is further configured to at least: recognize aplurality of points of the portion of the surface below the aerialvehicle in the first digital image by the at least one computerprocessor; recognize at least some of the plurality of points in thesecond digital image by the at least one computer processor; determineranges to the at least some of the plurality of points based at least inpart on the distance, the focal length, the first digital image and thesecond digital image; and generate a depth map based at least in part onthe ranges to the at least some of the plurality of points, wherein thealtitude of the aerial vehicle is determined based at least in part onthe depth map.
 5. A method comprising: capturing a first image of atleast a portion of a scene by a first imaging device at a first time,wherein the first imaging device is integrated into a first surface of afirst blade of a first propeller of a first aerial vehicle at a firstradius from a first hub of the first propeller, and wherein the firstpropeller is rotating at a first angular velocity about a first axis ofrotation at the first time; capturing a second image of at least theportion of the scene by the first imaging device at a second time,wherein the first propeller is rotating at the first angular velocityabout the first axis of rotation at the second time; determining adistance between a first position of the first imaging device at thefirst time and a second position of the first imaging device at thesecond time; and determining a first range to at least the portion ofthe scene based at least in part on the first image and the second imageby at least one computer processor.
 6. The method of claim 5, whereinthe first blade is in a first angular orientation at the first time,wherein the first blade is in a second angular orientation at the secondtime, and wherein the distance is determined based at least in part onthe first angular orientation, the second angular orientation and thefirst radius.
 7. The method of claim 5, further comprising: identifyinga first representation of at least one point of the scene in the firstimage by the at least one computer processor; and identifying a secondrepresentation of the at least one point of the scene in the secondimage by the at least one computer processor, wherein determining thefirst range to at least the portion of the scene comprises: defining afirst line from the first position through the first representation bythe at least one computer processor; defining a second line from thesecond position through the second representation by the at least onecomputer processor; determining a position of an intersection of thefirst line and the second line by the at least one computer processor;and determining the first range to the portion of the scene based atleast in part on the position of the intersection by the at least onecomputer processor.
 8. The method of claim 5, further comprising:determining a first angular orientation of the first blade at the firsttime; and determining a second angular orientation of the first blade atthe second time, wherein determining the first range to at least theportion of the scene comprises: aligning the first image and the secondimage with respect to one another based on a difference between thefirst angular orientation and the second angular orientation.
 9. Themethod of claim 5, further comprising: recognizing a plurality of pointsof the scene in the first image by the at least one computer processor;recognizing at least some of the plurality of points of the scene in thesecond image by the at least one computer processor, and whereindetermining the first range to the at least one point of the scenecomprises: determining ranges to the at least some of the plurality ofpoints of the scene based at least in part on the first image and thesecond image by the at least one computer processor; defining a rangemap for the scene based at least in part on the ranges, wherein therange map represents distances to each of a plurality of regions; anddetermining the first range based at least in part on the range map. 10.The method of claim 5, further comprising: selecting, by at least onecomputer processor, a landing site for the first aerial vehicle at thescene based at least in part on the first range.
 11. The method of claim5, further comprising: capturing a third image of at least the portionof the scene by a second imaging device at the first time or the secondtime, wherein the second imaging device is integrated into a secondsurface of a second blade of the first propeller at a second radius fromthe first hub; and determining a second range to at least the portion ofthe scene based at least in part on the third image and at least one ofthe first image or the second image by the at least one computerprocessor.
 12. The method of claim 5, further comprising: capturing athird image of at least the portion of the scene by a second imagingdevice at the first time or the second time, wherein the second imagingdevice is integrated into a second surface of at least one of a frame ofthe first aerial vehicle or a first motor rotatably coupled to the firstpropeller; and determining a second range to at least the portion of thescene based at least in part on the third image and at least one of thefirst image or the second image.
 13. The method of claim 5, furthercomprising: capturing a third image of at least the portion of the sceneby a second imaging device at one of the first time or the second time,wherein the second imaging device is integrated into a second surface ofa second blade of a second propeller of the first aerial vehicle at asecond radius from a second hub of the second propeller, and wherein thesecond propeller is rotating at a second angular velocity; anddetermining a second range to at least the portion of the scene based atleast in part on the third image and at least one of the first image orthe second image.
 14. The method of claim 5, wherein a differencebetween the first time and the second time is approximatelyone-hundredth of one second.
 15. The method of claim 5, wherein thefirst angular velocity is at least one thousand revolutions per minute.16. An aerial vehicle comprising: a frame; a propulsion motor mounted tothe frame, wherein the propulsion motor is configured to rotate a shaftabout an axis defined by the shaft; a propeller having a plurality ofblades, wherein the propeller is rotatably coupled to the shaft; a firstdigital camera integrated into one side of one of the plurality ofblades; a second digital camera integrated into at least one of theframe or an external surface of the propulsion motor; and at least onecomputer processor, wherein the at least one computer processor isconfigured to at least: cause the propulsion motor to rotate thepropeller about the axis at a predetermined speed; cause the firstdigital camera to capture a first digital image at a first time, whereinthe one of the plurality of blades having the digital camera embeddedtherein is aligned in a first angular orientation at the first time;cause the second digital camera to capture a second digital image atapproximately the first time; determine a distance between a firstposition of the first digital camera at the first time and a secondposition of the second digital camera at approximately the first timebased at least in part on the first angular orientation; determine thatat least one point is represented within each of the first digital imageand the second digital image; and in response to determining that the atleast one point is represented within each of the first digital imageand the second digital image, determine a range from the aerial vehicleto the at least one point based at least in part on the distance, afirst focal length of the first digital camera and a second focal lengthof the second digital camera.
 17. The aerial vehicle of claim 16,wherein the at least one computer processor is further configured to atleast: identify a first representation of the at least one point in thefirst digital image; identify a second representation of the at leastone point in the second digital image; define a first line from thefirst position through the first representation; define a second linefrom the second position through the second representation; determine aposition of an intersection of the first line and the second line; anddetermine the range to the at least one point based at least in part onthe position of the intersection.
 18. The aerial vehicle of claim 16,wherein the at least one computer processor is further configured to atleast: recognize a plurality of points in the first digital image;recognize at least some of the plurality of points in the second image;determine ranges to the at least some of the plurality of points of thescene based at least in part on the distance, the first focal length andthe second focal length; and define a range map for the scene based atleast in part on the ranges, wherein the range map represents distancesto each of a plurality of regions, wherein the range to the at least onepoint is determined based at least in part on the range map.
 19. Theaerial vehicle of claim 16, wherein the first imaging device isintegrated into an underside of the one of the plurality of blades,wherein the second imaging device has a field of view extending belowthe aerial vehicle, and wherein the range is an altitude of the aerialvehicle over at least a portion of a surface beneath the aerial vehicle.20. The aerial vehicle of claim 16, wherein the predetermined speed isat least one thousand revolutions per minute.